\newpage

\section{Introduction}

AI is the new electricity that brings about a big transform in
the society, ranging from commercial advertising, health care, 
auto driving, agriculture and many others. Deep Neural Networks 
demonstrate its impressive power in mathematical problems, computer 
vision, natural language processing etc. However, to most people 
these state-of-the-art neural networks remain to be black-boxes even 
they have successfully taken them into applications. Still, to bring
the potential of neural networks into full play, it is inevitable to
have a better command of what neural networks is doing and why neural
networks work so well?


\par This article aims to cast light on neural networks. In \autoref{sec:NN},
we first look into the foundation of neural networks including the
representation of a neuron, the process of forward propagation and 
back propagation, and the basic intuition of gradient descent (GD).
In \autoref{sec:Problem}, we briefly summarize the most common 
problems during training a neural network. Subsequently, in \autoref{sec:Solution},
we are going to introduce some effective tricks including a overview
of initialization and optimization, to overcome these problems. In
\autoref{sec:Question}, we take a short look at the three main parts in
neural networks theory, they cover: the power of approximation of 
deep neural networks, the landscape of the empirical risk and the 
generalization of GD.

